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Abstract 



In this paper, we consider the problem of robust joint source-channel coding over an additive white 
O . Gaussian noise channel. We propose a new scheme which achieves the optimal slope of the signal-to- 

distortion (SDR) curve (unlike the previously known coding schemes). Also, we propose a family of 
^ ' robust codes which together maintain a bounded gap with the optimum SDR curve (in terms of dB). To 

^ show the importance of this result, we drive some theoretical bounds on the asymptotic performance of 



delay-limited hybrid digital-analog (HDA) coding schemes. We show that, unlike the delay-unhmited 



l/^ ■ case, for any family of delay-limited HDA codes, the asymptotic performance loss is unbounded (in 

' terms of dB). 

o 

: I. Introduction 

5r , In many applications, delay-limited transmission of analog sources over an additive white 

Gaussian noise channel is needed. Also, in many cases, the exact signal-to-noise-ratio (SNR) is 
not known at the transmitter, and may vary over a wide range of values. Two examples of this 
scenario are transmitting an analog source over a quasi- static fading channel and/or multicasting 
it to different users (with different channel gains). 

Without considering the delay limitations, digital codes can theoretically achieve the optimal 
performance in the Gaussian channel. Indeed, for the ergodic point-to-point channels. Shannon's 
source-channel coding separation theorem [1] [2] ensures the optimality of separately designing 
source and channel codes. However, for the case of limited delay, several articles [3] [4] [5] 
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[6] [7] have shown that joint source-channel codes have a better performance as compared to 
the separately designed source and channel codes (which are called tandem codes). Also, digital 
coding is very sensitive to the mismatch in the estimation of the channel SNR. 

To avoid the saturation effect of digital coding, various analog and hybrid digital-analog 
schemes are introduced and investigated in the past [8]-[23]. Among them, examples of l-to-2- 
dimensional analog maps can be found as early as the works of Shannon [8] and Kotelnikov [9] 
and different variations of Shannon-Kotelnikov maps (which are also called twisted modulations) 
are studied in [10] [11] [19]. Also, in [14] and [15], analog codes based on dynamical systems 
are proposed. Although these codes can provide asymptotic gains (for high SNR) over simple 
repetition codes, they suffer from a threshold effect. Indeed, when the SNR becomes less than 
a certain threshold, the performance of these systems degrades severely. Therefore, design 
parameters of these methods should be chosen according to the operating SNR, resulting in 
sensitivity to SNR estimation errors. Also, although the performance of the system is not saturated 
for the high SNR values (unlike digital codes), the scaling of the end-to-end distortion is far 
from the theoretical bounds. Theoretical bounds on the robustness of joint source channel coding 
schemes (for the delay-unlimited case) are presented in [24] and [25]. 

To achieve better signal-to-distortion (SDR) scaling, a coding scheme is introduced in [26] 
[27] which uses B repetitions of a {k,n) binary code to map the digits of the infinite binary 
expansion of k samples of the source to the digits of a nS-dimensional transmit vector. For 
this scheme, the bandwidth expansion factor is rj — ^ and the SDR asymptotically scales as 
SDR oc SNR^, while in theory, the optimum scaling is SDR oc SNR^. Thus, this scheme 
cannot achieve the optimum scaling by using a single mapping. 

In this paper, we address the problem of robust joint source-channel coding, using delay- 
limited codes. In particular, we show that the optimum slope of the SDR curve can be achieved 
by a single mapping. The rest of the paper is organized as follows: 

In section 11, the system model and the basic concepts are presented. Section III presents an 
analysis of the previous analog coding schemes, and their limitations. In section IV, we introduce 
a class of joint source-channel codes which have a self-similar structure, and achieve a better 
asymptotic performance, compared to the other minimum-delay analog and hybrid digital-analog 
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coding schemes. The asymptotic performance of these codes, in terms of the SDR scaling, is 
comparable with the scheme presented in [26], but with a simpler structure and a shorter delay. 
We investigate the limits of the asymptotic performance of self-similar coding schemes and their 
relation with the Hausdorff dimension of the modulation signal set. In section V, we present 
a single mapping which achieves the optimum slope of the SDR curve, which is equal to the 
bandwidth expansion factor. Although this mapping achieves the optimum slope of the SDR 
curve, its gap with the optimum SDR curve is unbounded (in terms of dB). In section VI, we 
construct a family of robust mappings, which individually achieve the optimum SDR slope, and 
together, maintain a bounded gap with the optimum SDR curve. We also analyze the limits on 
the asymptotic performance of the delay-limited HDA coding schemes. 

II. System model and theoretical limits 

We consider a memoryless {X,i}°l^ uniform source with zero mean and variance j^, i.e. 
— I < a;^ < |. Also, the samples of the source sequence are assumed independent with identical 
distributions (i.i.d.). Although the focus of this paper is on a source with uniform distribution, 
as it is discussed in Appendix C, the asymptotic results are valid for all distributions which have 
a bounded probability density function. 

The transmitted signal is sent over an additive white Gaussian noise (AWGN) channel. The 
problem is to map the one-dimensional signal to the A^-dimensional channel space, such that 
the effect of the noise is minimized. This means that the data x, — | < x < |, is mapped to the 
transmitted vector s = (si, sat)- At the receiver side, the received signal is y = s + z where 
z— (zi, zn) is the additive white Gaussian noise with variance u^. 

As an upper bound on the performance of the system, we can consider the case of delay- 
unlimited. In this case, we can use Shannon's theorem on the separation of source and channel 
coding. By combining the lower bound on the distortion of the quantized signal (using the rate- 
distortion formula) and the capacity of parallel Gaussian channels with the noise variance o"^, 
we can bound the distortion D — E{\x — xp} as [15] 

D > ca^"^ (1) 

where c is a constant number. 
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III. Codes based on dynamical systems and hybrid digital-analog coding 

Previously, two related schemes, based on dynamical systems, have been proposed for the 
scenario of delay-limited analog coding: 

1) Shift-map dynamical system [14] 

2) Spherical shift-map dynamical system [15] 
These are further explained in the following. 

A. Shift-map dynamical system 

In [14], an analog transmission scheme based on shift-map dynamical systems is presented. 
In this method, the analog data x is mapped to the modulated vector (si, sat) where 

Si = X mod 1 (2) 

Sj+i = biSi mod 1, for 1 < 2 < — 1 (3) 

where 6j is an integer number, bi >2. The set of modulated signals generated by the shift map 
consists of 6i • ^2 ■ ••• ■ ^tv-i parallel segments inside an A^-dimensional unit hypercube. In [15], 
the authors have shown that by appropriately choosing the parameters {b^} for different SNR 
values, one can achieve the SDR scaling (versus the channel SNR) with the slope N — e, for 
any positive number e. Indeed, we can have a slightly tighter upper bound on the end-to-end 
distortion as follows: 

Theorem 1 Consider the shift-map analog coding system which maps the source sample to an 
N -dimensional modulated vector. For any noise variance^ < |, we can find parameter a 
such that for the shift-map scheme with the parameters bi = a> 2, the distortion of the decoded 
signal D is bounded aj^ 



D<ca2^(-loga)^"i (4) 



where c depends only on N. 



'The result is still valid if < S, for some < 5 < 1 (but c will depend on S). 
^We use log x to denote the natural logarithm, i.e. log^ x. 
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Proof: See Appendix A. ■ 

Also, we have the following lower bound on the end-to-end distortion: 

Theorem 2 For any shift-map analog coding scheme and any noise variance < |, the output 
distortion is lower bounded as 

Z^>cV^(_loga)^-i (5) 

where d depends only on N. 
Proof: See Appendix B. ■ 

B. Spherical shift-map dynamical system 

In [15], a spherical code based on the linear system = Asy is introduced, where st 
is the 2iV-dimensional modulated signal and A is a skew-symmetric matrix, i.e. A^ = —A. 
This scheme is very similar to the shift-map scheme. Indeed, with an appropriate change of 
coordinates, the above modulated signal can be represented as 

St = —;= (cos27rx, cos2a7rx, cos2a^~^-yrx, 

sin27rx, sin2a7rx, sin2a^^"'^7rx) (6) 

for some parameter a. 

If we consider Sgm as the modulated signal generated by the shift-map scheme with parameters 
6j = a in (|3]), then, ^ can be written in the vector form as 

St= {Re{ e^^"=™ },Im{ e^^^^'" } ) . (7) 

The relation between the spherical code and the linear shift-map code is very similar to the 
relation between phase- shift-keying (PSK) and pulse- amplitude-modulation (PAM). Indeed, the 
spherical shift-map code and PSK modulation are, respectively, the linear shift-map and PAM 
modulations which are transformed from the unit interval, |), to the unit circle. 
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Fig. 1. The shift-map modulated signal set for N — 3 dimensions and a — 2. 



For the performance of the spherical codes, the same resuk as Theorem 1 is valid. Indeed, 
for any parameters a and A^, the spherical code asymptotically has a saving of ^-^2 ^-^^ 
in the power. This asymptotic gain results from transforming the unit-interval signal set (with 
length 1 and power j^) to the unit-circle signal set (with length 27r and power 1) . However, 
the spherical code uses 2N dimensions (compared to N dimensions for the linear shift-map 
scheme). 

For both these methods, for any fixed parameter a, the output SDR asymptotically has linear 
scaling with the channel SNR. The asymptotic gain (over the simple repetition code) is approxi- 
mately a^*^^~^) (because the modulated signal is stretched approximately a^^^ timesjj. Therefore, 
a larger scaling parameter a results in a higher asymptotic gain. However, by increasing a, the 
distance between the parallel segments of the modulated signal set decreases. This distance is 
approximately ^ and for the low SNRs (when the noise variance is larger than or comparable to 
^), jumping from one segment of the modulated signal set to another one becomes the dominant 
factor in the distortion of the decoded signal which results in a poor performance in this SNR 
region. Thus, there is a trade-off between the gain in the high-SNR region and the critical noise 

^The exact asymptotic gain is equal to the scaling factor of the signal set, i.e. a'^'^^^' (^1 + ^ + ••■ H — ^(iv-i) ) fo'' '^e shift 
map and ■^2z^(j2{]v-i) / ^ _j — j ([jg spherical shift map. 
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level which is fatal for the system. By increasing the scaling parameter a, the asymptotic gain 
increases, but at the same time, a higher SNR threshold is needed to achieve that gain. In [28], 
the authors have combined the dynamical-system schemes with LDPC and iterative decoding to 
reduce the critical SNR threshold. However, overall behavior of the output distortion is the same 
for all these methods. Also, in [29] and [30], a scheme is introduced for approaching arbitrarily 
close to the optimum SDR, for colored sources. However, it is not delay-limited and it only 
works for the bandwidth expansion of 1. 

The shift-map analog coding system can be seen as a variation of a hybrid-digital-analog 
(HDA) joint source-channel code. Various types of such hybrid schemes are investigated in 
[16] [17] [18] [24] and [31]. Indeed, for the shift-map system, we can rotate the modulated 
signal set such that all the parallel segments of it become aligned in the direction of one of the 
dimensions. In this case, by changing the support region of the modulated set (which is a rotated 
A^-dimensional cube) to the standard cube, we obtain a new similar modulation which is hybrid 
digital-analog and has almost the same performance. In the new modulation, the information 
signal is quantized by a^~^ points in an (A'^ — 1) -dimensional sub-space and the quantization 
error is transmitted over the remaining dimension. 

Regarding the scaling of the output distortion, the performance of the shift-map scheme, with 
appropriate choice of parameters for each SNR, is very close to the theoretical limit. In fact, 
the output distortion scales as a'^^{— logo")^^^, instead of being proportional to cr^^. However, 
for any fixed set of parameters, the curve of SDR-versus-SNR (in dB) is saturated by the unit 
slope (instead of N). This shortcoming is an inherent drawback of schemes like the shift-map 
code or the spherical code (which are based on dynamical systems). Indeed, in [32], it is shown 
that no single differentiable mapping can achieve an asymptotic slope better than 1. This article 
addresses this shortcoming. 

There are some other analog codes in the literature which use different mappings. Analog 
codes based on the 2-dimensional Shannon map [20] [21] [22] [23], or the tent map [14] are 
examples of these codes. However, all these codes share the shortcomings of the shift-map code. 
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IV. Joint source-channel codes based on fractal sets 

In this section, we propose a coding scheme, based on the fractal sets, that can achieve slopes 
greater than 1 (for the curve of SDR versus SNR). 

Scheme I: For the modulating signal x, — | < x < we consider the binary expansion of 

X + 2" 



x+-= (O-hhh...)^. (8) 

Now, we construct si, S2, sn as 

si = (O ■ bibN+ib2N+i---)^ (9) 

S2 = (O ■ 62W2^27V+2---)„ (10) 



Stv = (O ■ hNh2NhzN--)^ (11) 

where (O ■ 6162^3 •••)^ is the base-a expansioj^. 

Theorem 3 In the proposed scheme, for any a > 2 and noise variance cr^ < \, the output 
distortion D is upper bounded by 

D <ca^\-\ogaf (12) 
where c depends only on N, and (3 = N^^. 

Proof: Consider Zi as the Gaussian noise on the ith dimension: 

Pr I l^il > 2^/Na^/-\oga^ = (13) 

''in this article, we define the base-Q expansion, for any real number a > 2 and any binary sequence (6162&3...), as 



{0-hb2b3...)^^j:Z,b.a-\ 
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Now, we bound the distortion, conditioned on l^;^! < 2\fNa\/— logo for 1 < i < iV. If the 
k\h digit of Si and s- are different, 

(15) 



\si-s'i\ > 



0-^1000.. - 0-^0111... 

fe-1 / V k-1 



(16) 



(17) 



Therefore, if \si — s\\ < S for S > 0, the first k digits of Si and s- are the same, where 
k > [— log^ (^^iri)] ~ Now, by considering 5 — Ay/N a^J— log a. 



\si - s\\ < 2\zi\ < 4:VNa^y-\oga 



(18) 



k> - log^ 



Ay/Na^/- log a 



Therefore, for 1 < i < TV, the first 



a - 2 



4:VNa^/-\oga 



- 1 



a-2 

digits of Si, S2, Sat can be decoded without any error, hence, the first 



TV 



logo 



4:VNay/-\oga 
a-2 



1 



(19) 



bits of the binary expansion of x can be reconstructed perfectly. In this case, the output distortion 
is bounded by 



(20) 



D < Ci(7^^(-l0g(7) 



N 



(21) 



where ci depends only on a and A^. By combining the upper bounds for the two cases, noting 
that a <1 
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N 

D < ^Pr > 2^/Nay^-\oga^ + cia^^{-\oga)^ (22) 

i=l 

<(72^ + ci(72^(-loga)^ (23) 
< c(t2^(- log (j)^. (24) 

■ 

According to the theorem 2, for any e > 0, we can construct a modulation scheme that 
achieves the asymptotic slope of iV — e (for the curve of SDR versus SNR, in terms of dB). As 
expected (according to the result by Ziv [32]), none of these mappings are differentiable. More 
generally, Ziv has shown that [32]: 

Theorem 4 ( [32], Theorem 2) For the modulation mapping s — f{x), define 

df{A) = E{\\f{x + A)-f{x)r}. 
If there are positive numbers A, 7, Aq such that 

df{A) < AA^ for A < Aq. (25) 
Then, there is constant c such that 

D>ca^. (26) 

In Scheme I, by decreasing a, we can increase the asymptotic slope (3. However, it also 
degrades the low-SNR performance of the system. This phenomenon is observed in figure 3. 

In scheme I, the signal set is a self-similar fractal [33], where the parameter (3, which 
determines the asymptotic slope of the curve, is the dimension of the fractal. There are different 
ways to define the fractal dimension. One of them is the Hausdorff dimension. Consider T 
as a Borel set in a metric space, and 21 as a countable family of sets that covers it. We 
define H^{T) — inf ^^^g^ (diameter(^))'*, where the infimum is over all countable covers 
that diameter of their sets are not larger than e. The s-dimensional Hausdorff space is defined 
as H^{!F) — \\m.s^QHl{T) — sup^^Q H^{J^). It can be shown that there is a critical value sq 
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such that for s < sq, this measure is infinite and for s > sq, it is zero [33]. This critical value 
So is called the Hausdorff dimension of the set JF. 

Another useful definition is the box-counting dimension. If we partition the space into a grid 
of cubic boxes of size e, and consider as the number of boxes which intersect the set JF, the 
box-counting dimension of JF is defined as 

Dim,(^)=limi^ (27) 

e^O log- 

It can be shown that for regular self-similar fractals, the Hausdorff dimension is equal to 
the box-counting dimension [33]. Intuitively, theorem 3 means that in scheme I among the 
available dimensions, only P dimensions are effectively used. Indeed, we can show that for any 
modulation sej^ with box-counting dimension /3, the asymptotic slope of the SDR curve is at 
most /3: 

Theorem 5 For a modulation mapping s = /(x), if the modulation set T has box-counting 
dimension (3, then 

lim < 2/3. (28) 

o-^o logcr 

Proof: We divide the space to boxes of size a. Consider m„ as the number of cubic boxes that 
cover T. We divide the source signal set to Anicr segments of length Consider ^i, ^4^,^ 
as the corresponding iV-dimensional optimal decoding regions (based on the MMSE criterion), 
and Bi, Bim^ as their intersection with the m„ cubes (see figure 2). Total volume of these 4mo^ 
sets is equal to the total volume of the covering boxes, i.e. m^a^ . Thus, at least, half of these 
sets (i.e. 2ma- of them) have volume less than |cr^. For any of these sets such as Bi and any box, 
the volume of the intersection of that box with the other sets is at least Vmin = cr^ — |cr^ = 
For any point in the corresponding segments of the set Bi, the probability of decoding to a wrong 
segment is lower bounded by the probability of a jump to the neighboring sets in the same box. 
Because the variance of the additive Gaussian noise is cr^ per each dimension, and for such a 
jump the squared norm of the noise at most needs to be Na'^ (square of the diameter of the 
box), the probability of such a jump to the neighboring sets can be lower bounded as 

^Modulation set is the set all possible modulated vectors. 
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Fig. 2. Boxes of size a and their intersections with the decoding regions 



Pr(j«mp) > Vrnin ' mill (z) (29) 
||zp<ArCT2 

1 AT" 1 jVo-^ 1 _iV 

- ■ T^TT^^^ ^ ^ ^iv,, 2 , (30) 

2 (27r)^(7^ 2T+i7r2- 

where (z) is the pdf of the noise vector z. 

Now, for these segments of the source, consider the subsegments with length at the center 
of them. When the source belongs to one of these subsegments, wrong segment decoding results 

in a squared error of at least ■ i^^^ Iowt)) ~ (lom") • Thus, for these subsegments 

whose total length is at least ■ 2m„ = ^^j, at least with probability Fr (jump), we have a 



squared error which is not less than ^^q^^ • Therefore, 



C>lprO«mp).(^) =^ (31) 
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where c only depends on the bandwidth expansion N. On the other hand, based on the definition 
of the box-counting dimension, 

logm^ 

/3 = lini-j —. (32) 



By using dSB and dSl]), 



lim < 2/3. (33) 

o-^O log CT 



It should be noted that theorem 5 is valid for all signal sets, not just self-similar signal sets. 
As a corollary, based on the fact that the box-counting dimension can not be greater than the 
dimension of the space [33], Theorem 5 provides a geometric insight to ©• 

Another scheme based on self- similar signal sets and the infinite binary expansion of the 
source is proposed in [26] [27], which similar to the scheme proposed in this section, achieves a 
SDR scaling better than linear coding, but cannot achieve the optimum SDR scaling. The scheme 
presented in [26] is based on using B repetitions of a {k,n) binary code to map the digits of the 
infinite binary expansion of k samples of the source to the digits of a nS-dimensional transmit 
vector. This scheme shares the shortcoming of Scheme I. In [26], the bandwidth expansion factor 
is = ^ and the SDR asymptotically scales as SDR oc SNR^, instead of the optimum scaling 
SDR oc SNR^. The main difference between Scheme I and the scheme proposed in [26] is 
that in Scheme I, the delay is minimum (it uses only one sample of the source for coding), 
but in [26], the delay is k, and the the ratio between the SDR exponent and the optimum SDR 
exponent is dependent on the delay (it is ^), i.e. to increase it, one needs to increase the length 
of the binary code, which results in increasing the delay. 

The idea of using the infinite binary expansion of the source, for joint source-channel coding, 
can be traced back to Shannon's 1949 paper [8], where shuffling the digits is proposed for band- 
width contraction (i.e. mapping high-dimensional data to a signal set with a lower dimension). 
For bandwidth expansion, space-filling self-similar signal sets have been investigated in [13], 
however, the SDR scaling of those schemes are not better than linear coding. The reason is 
that when we use a self-similar set to fill the space, the squared error caused by jumping to 
adjacent subsets dominates the scaling of the distortion. To avoid this effect, we need to avoid 
filling the whole space. This results in losing dimensionality for self-similar sets, which results 
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in sub-optimum SDR scaling (as investigated in this section). To avoid this drawback, we need 
to consider signal sets which are not self-similar, as proposed in the next section. 

V. Achieving the optimum asymptotic SDR slope using a single mapping 

Although Scheme I can construct mappings that achieve near-optimum slope for the curve of 
SDR (versus the channel SNR), none of these mappings can achieve the optimum slope A^. To 
achieve the optimum slope with a single mapping, we slightly modify Scheme I: 

For the modulating signal x, consider x + | = (0.bib2b-i...)^. We construct si, S2, sn as 

si = (^O.biOb N(N+i) ^ -^& iv(iv+i) ^ ^...6 iv(iv+i) ^ ^lOb {2N)(2^N+l) ii"-)^ (34) 



S2 — ^0.62^30^ ^-^b (N+l)(N+2) ^^...b (N+l)(N+2) ^ J^T^ ' ' ■) ^ ^^^^ 



Sjv = I 0.b iv(iv~i) I -I b jv(iv-i) , ^...& jv(jv+i) 0... I (36) 

V 2 '^^ 2 2 / 2 

The difference between this scheme and Scheme I is that instead of assigning the kN + zth bit 
to the signal Sj, the bits of the binary expansion of x + | are grouped such that the Ith group 
(I = kN + i) consists of I bits and is assigned to the zth dimension. In decoding, we find the 
point in the signal set which is closest to the received vector s + z. If \zi\ < 2^^^^fe=o (kN+i+i)^ 
the first X]fc=o (^-^ + i + 1) bits of Si can be decoded error-freely (for 1 < i < N) which include 
Ylk=o + ^) ^^^^ °f source x. 

Theorem 6 Using the mapping constructed by Scheme II, for any noise variance < |, the 
output distortion D is upper bounded by 



D < ciCj2^2'=V-i°S2'^ (37) 



where ci and C2 only depen 



on N. 



^Throughout this paper, Ci,C2, ... are constants, independent of a (they may depend on A'^). 
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Proof: Let Zi be the Gaussian noise on the ith dimension and assume that n is selected such 
that 



n+l 



n+2 



^ (A;iV + 1) < _ iog2 a < ^ (A;iV + 1). 



k=l 



The probability that l^i] > 2 ^ Efe=i(fc^+i) is negligible. Indeed, 



(38) 



n+l 



-log2a>5^(A;iV+l) \ < 



k=l 



2Q 



2-ES(fcA^+i) 



2Q (2("+i)^) 



< 2Q (2V i:.- 
2Q (^2V-i°g2-j < 2 



fcAT + l 



-2^V^ log2 < 



where (a) because a / ^^^^ A;A^ + 1 



jV(n+2)(n+3) 



(39) 

(40) 
(41) 
(42) 



+ n + 2 < (n + l)iV for iV > 2, and (6) 



because (|38l) , and (c) because (5(x) < |e "2 . 

On the other hand, when IzA < 2-^~^"k=^^^^+^\ for 1 < i < N, IzA < 



hence the first Ylk=o + 2 + 1) bits of Sj can be decoded error-freely which include Ylk=o + 
bits of the source x. Thus, the first J2iLi J22Zoi^^ + i) = YTj=ij bits of x can be decoded 
error-freely. Now, 



nN 



nN{nN + 1) 



(43) 



^^_^^- iV(n + 2)(n + 3) ^ ^ ^ N' jbn + 6) + nN + AN ^^^^ 



N[J2ikN + l] 



N^ (5n + 6) + nN + 4:N 



.k=l 



(45) 



>N[J2ikN+l)]-cs> 



.k=l 



n+l 



\ k=l 



(46) 
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where C3 depends only on A^. Therefore, by using the assumption (l38l) . 

nN 

J2j> (47) 



- log2 <T - C3 a/- log2 0- (48) 
Consequently, the output distortion is bounded by 

N 

D < 5^Pr{|^,| > 2-1-^^-'=^} + 2-2S""i^- (49) 
j=i 

< 2Q ^2V^^^°^20"j _|_ 22Af log2 (T+2c3 ^- log2 o- ^^Q^ 

= 2Q (^2V-i°g2'^j + ^27V2C2V-i°g2<^, (51) 



^ D< cict2^2^2V~'°S2'^. (52) 

■ 

It should be noted that in this proof, the assumption of having a uniform distribution is not 
used, and the above proof is valid for any source whose samples are in the interval [— |, |)- 
In Appendix C, we extend the scheme proposed in this section to other sources which are not 
necessarily bounded. 

VI. Approaching a near-optimum SDR by delay-limited codes 

In [24], a family of hybrid digital-analog (HDA) source-channel codes are proposed which 
together can achieve the optimum SDR curve and each of them only suffers from the mild 
saturation effect (the asymptotic unit slope for the curve of SDR versus SNR). However, their 
approach is based on using capacity-approaching digital codes as a component of their scheme. 
In [25], it is shown that for any joint source-channel code that touches the optimum SDR curve 
at a certain SNR point, the asymptotic slope can not be better than one. 



17 



In this section, we consider the problem of finding a family of delay-limited analog codes 
which together have a bounded asymptotic loss in the SDR performance (in terms of dB). Results 
of Section EI show that none the previous analog coding schemes (based on dynamical systems) 
can construct such a family of codes. In this section, we also show that no HDA source-channel 
coding scheme can achieve this goal. 

In the HDA source-channel coding, in general, to map an M dimensional source to an N 
dimensional signal set, the source is quantized by k points which are sent over N—M dimensions 
and the residual noise is transmitted over the remaining M dimensions. In other words, the region 
of the source (which is a hypercube for the case of a uniform source) is divided into k subregions 
Ai, ■..■,Ak- These subregions are mapped to k parallel subsets of the N dimensional Euclidean 
space, .A'l, A!^, where A!^ is a scaled version of Ai with a factor of a. 

Theorem 7 Consider a HDA joint source-channel code which maps an M-dimensional uniform 
source (inside the unit cube) to k parallel M-dimensional subsets of an N dimensional Euclidean 

space (N > M), with a power constraint of 1. If the decoding of digital and analog parts are 
done separately, for any noise variance < 1, the output distortion is lower bounded by 

2N N-M 

D> CO M log a) M (53) 
where c depends only on M and N. 
Proof: See Appendix D. ■ 

Now, we construct families of delay-limited analog codes which by a proper choice of param- 
eters (according to the channel SNR) have a bounded asymptotic loss in the SDR performance 
(in terms of dB). 

Type I - Family of piece-wise linear mappings: For any 2~'^~^ < (J < 2 *^, for /c > 0, we 
construct an analog code as the following: 
For x + I = (0 • 

bib2---bNk-i)2 "I 2^*^-1 — ' where {•} represents the fractional part, we 

construct Si, S2, sn as 

k 

1=1 
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S2 = $^(2-^ + 2-*^(A;-z))%_i)^+2 



i=l 

k—1 [ r\Nk—l \ 



2Nk-l 

1=1 



(54) 



First, we show that < sj < 2, for 1 < j < N. By using the fact that the value of the bits 
are at most 1, and |2^'^~^a:} < 1, 



k+l 



< J](2-* + 2-''(k - i)) + 2-*=-^ = ^ 2-^ + 2-*= - (55) 



i=l i=l 



<l + 2-fe >(^^ ^) <2. (56) 

Therefore, noting that < Sj < 2, by an appropriate shift (e.g. modifying the transmitted signal 
set as s' = s— 1), the transmitted power can be bounded by one. Next, we show that the proposed 
scheme has a bounded gap (in terms of dB) to the optimum SDR curve: 

Theorem 8 In the proposed scheme, noise variance < |, the output distortion D is upper 
bounded by 

D < ca'''' (57) 

where c depends only on N. 

Proof: The signal set consists of 2^^~^ segments of length 2~^~^, where each of them is a 
subsegment of the source region (the unit interval), scaled by a factor of 2^'^~*^~^. 

The probability that the first error occurs in the lih. bit {I = [i — 1)N + j, where 1 < j < A'^) 
of X is bounded hy Pi <2Q —'^Q {2 ~ 2n) results in an output squared error of at 
most Di < 4"'+^ = 4-{»-i)^-j+i_ Therefore, by considering the union-bound over all possible 
errors, we obtain 
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D<J2 Di-Pi + D 

no— bit— e 



1=1 

Nk-l 



< 



5] 4-'+^ .2q(^-±)+ 4-(^'=-^-^)a^. (58) 
1=1 ^ ^ 



2 

Now, by using Q{x) < e"^ and 2~^~^ < a, we have 



fciV-l 



=1 

fciV-l 



=1 



- ("-'/y^^) ^ 4-(iVfc-fc-2)^2 



2-2.7V . 23 . 28iv^ ^ 2- ^-'/^;--"^ +4- 



/=1 



oo „ 
(fc-i/JV-8jV)^ 



^ 2-2fe''v . 23 . 2^^^ ^ 2- 8 + 4-(''Vfe-fe-2)^2 



l=—oo 



2-2kN _ 23 . 2^^'^ ^ ^_(jvit-ik-2)^2 
Z'=-oo 



< 2-2^^ . ci + 4-(^^-'=-2)a2 

< ccT^^. (59) 

■ 

It is worth noting that in the proposed family of codes, for each code, the asymptotic slope of 
the SDR curve is 1 (as we expected from the fact that for each code, the mapping is piecewise 
differentiable). We can mix the idea of this scheme with Scheme II of the previous section, to 
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construct a family of mappings where for each of them, the asymptotic slope is N, and together, 
they maintain a bounded gap with the optimal SDR (in terms of dB): 

Type II - Family of robust mappings: For x + ^ = {O ■ bib2b3...)^, we construct fk{x) = 
{si,S2,...,sn) as 

k 

k 

S2 = Xl'^^'^ + ^'^''^^ ~ ^))%-l)iV+2 + 2"*^-^ (O ■ bkN+2bkN+^Qb,,^ I (;v+iKiV+2) , 

i=l ^ ^ 

A; 

i=l 

Theorem 9 /n f/ie proposed family of mappings (Type II), there are constants c, Ci, C2, indepen- 
dent of a and k (are only dependent on N) such that for every integer k > 0, if we use the 
modulation map fk{x), 
i) For 2-^=-^ <a < 2"^ 



D <ca 



2N 



(60) 



ii) for any a <2 



-k-l 



D < cia^^2^^y/-^. (61) 

Proof: i) The probability that the first error occurs in the Ith bit (I — {i — 1)N + j < kN) 
of X is bounded by Pi > 2Q (^) and it results in an output squared error of at most 4~'+^, 
and when there is no error in the first Nk bits, the squared error is D' < 4"^^^. Therefore, by 
considering the union-bound over all possible errors, we have 

TV A: 

D<J2Di-Pi + D' 

1=1 
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'k I 



+ 4 



1\ K / 

Similar to the proof of theorem 8, by using Q{x) < and 2^^^^ < cr < 2^^^, we have 



2Ar 



«=1 



ii) Consider Zi as the Gaussian noise on the «th channel and assume that n is selected such 
that 



n+l 



n+2 



A; + ^(/A^+l) <-log2a< A; + ^(ZA^ + 1) (62) 
«=i z=i 

The probability that \zi\ > 2-^-^-^"=^^^^+^^ is negligible (it is bounded by 2Q (2("+^)^)). 

On the other hand, when \zi\ < 2~^'-^-S"=i ('^+^\ the first k + ^"Jg^ (/A^ + i + 1) bits of Si 

can be decoded error-freely (l < i < N) which include k + X]r=o^ + bits of x. Thus, the 

first /cA^ + Ei=ii bits of a; can be decoded error-freely. Now, similar to the proof of theorem 6, 



riN 



n+2 



N lk + J2i^N+l)\ -C5. 



n+2 



n+l 



\ «=1 



N ik + Y,ilN + 1) 



C6 



n+l 



A; + ^(/A^ + l) 



(63) 



(64) 



(65) 



1=1 



Therefore, by using the assumption (|62l) . 



nN 



(66) 
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- N l0g2 (7 - C6 ^/^]0g^ (67) 

Therefore, the output distortion is bounded by 

D < 2-2('=^+S"=i^) + 2Q (2(-+i)^) (68) 



=^ D < ci(7^^2'^V-i°g2'^. (70) 



VII. Simulation results 

In figure 3, for a bandwidth expansion factor of 4, the performance of Scheme I (with 
parameters a = 3 and 4) is compared with the shift-map scheme with a = 3. As we expect, 
for the shift-map scheme, the SDR curve saturates at slope 1, while the new scheme offers 
asymptotic slopes higher than one. For the proposed scheme, with parameters ai = 3 and 
a2 — 4, the asymptotic slope is respectively /3i = and f32 — = 2 (as expected from 
Theorem 3). Also, we see that the proposed scheme provides a graceful degradation in the low 
SNR region. 

Figure 4 shows the performance of Scheme II for A'^ = 4 dimensions. As it is shown in 
the figure, the asymptotic exponent of the SDR is close to the optimum value of 4, i.e. the 
bandwidth expansion ratio. The fluctuations of the slope of the curve is due to the fact that 
groups of consequent bits are assigned to each dimension, and for different ranges of SNR, 
errors in different dimensions become dominant (for example, for SNR values around 40-50dB, 
the error in the second layer of bits of Si becomes dominant in the overall squared error). By 
modifying Scheme 11 and assigning groups of bits of length I' = i + k{N — 1) (instead of 
I = i + kN) to the zth dimension, we can slightly improve the performance in the middle SNR 
range. Asymptotic exponents of the SDR in both variations of Scheme II are the same. 
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Fig. 3. The output SNR (or SDR) for the first proposed scheme (with a = 4 and 3) and the shift-map scheme with a — 3. 
The bandwidth expansion is N = 4. 



VIII. Conclusions 

To avoid the mild saturation effect in analog transmission (i.e. achieving the optimum scaling 
of the output distortion), one needs to use non-differentiable mappings (more precisely, mappings 
which are not differentiable on any interval). Two non-differentiable schemes are introduced in 
this paper. Both these schemes, which are minimum-delay schemes, outperform the traditional 
minimum-delay analog schemes, in terms of scaling of the output SDR. Also, one of them 
(Scheme II) achieves the optimum SDR scaling with a simple mapping (it achieves the asymptotic 
exponent N for the SDR, versus SNR). 
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Fig. 4. Performance of Scheme II for N — 4 dimensions, (a) corresponds to the scheme introduced in Section V and (b) 
corresponds to the other variation of Scheme II, when groups of I' = i + k{N — 1) bits are considered. 



Appendix A: Proof of Theorem 1 

The set of modulated signals consists of a^^^ parallel segments where the projection of 
each of them on the ith dimension has the length a~^^~^\ hence, each segment has the length 



Vl + + ••• + a^^(^^-^). By considering the distance of their intersections with the hyperspace 
orthogonal to the Nth dimension (which is at least a^^) and the angular factor of these segments, 
respecting to the SAr-axis, because a > 2, we can bound the distance between two parallel 
segments of the modulated signal set as (see Fig. 1) 

d > , > > (71) 

~ Vl + + ... + a-2(^-i) ~ Vl + 2-2 + ... + 2-2(^-1) " 2 

First, we consider the case of a^/^\oga < j^^- Consider a = -^-j^--^^^= . Probability of 



Sv^o-V-logcr 



a jump to a wrong segment (during the decoding) is bounded by 
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Pi(jump) <2Q{^] <2q(^] (72) 



<2Q|?^^^^^|. (73) 



By using Q{x) < 2 , 



Pr{jump) < e ^'^^T^^ = e2Jvioga ^ ^2jv_ ^^4^ 

On the other hand, each segment of the modulated signal set is a segment of the source signal 
set, stretched by a factor of a^~^Vl + + •■• + a~^(-^~^) (its length is changed from 
to Vl + + ... + a~'^^^~^^). Therefore, assuming the correct segment decoding, the average 
distortion is the variance of the channel noise divided by (^a^~^Vl + a"^ + ... + a~^(^~^)j : 

E {^\x — xf\no jump} — (75) 



(a^-Wl + a-2 + ... + a-2(^-i)y 



< (76) 



a 



2 2 

^ 2Ar / 1 \N-l ,rjrj^ 

2iN^ = 1 - ^~ ^) (77) 

8v^crV-logo-_ 

where x is the estimate of x and ci is independent of a and (T and only depends on N. Now, 
because E{|5; — x\'^\jump} and Pr {no jump) are bounded by 1, 

= Pr{jump) • E {|x — xj^ljump} + Pr(no jump) • E {|x — a:|^|no jump} (78) 
=^ L> < Pi{jump) + E {|x — a:|^|no jump] (79) 



< C2a^^(-log(j)^"\ for aV-log(7 < (80) 
On the other hand, for a^^— log a > j^^. 
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D < 1 = (7-^^(-loga)-(^-^) • a'^'i-logaf-' (81) 

= (av'^b^)"''^ • (-log a) • a'^'i-logaf-' (82) 

< • (-logx/2) . a^^(-loga)^-^ (83) 

Kcsa'^'i-logaf-K (84) 
Therefore, by combining these two bounds together, we obtain 



D<ca'''{-\ogay'-'. (85) 
Appendix B: Proof of Theorem 2 



We consider two cases: 

4 



Case 1) a < , , : 

Each segment of the modulated signal set is a segment of the source signal set, scaled by a 
factor of a^-Vl + a'^ + ... + a-2(^-i), hence 

D >E \^\x — x\'^\no jump] (86) 



(a^-Wl + + ... + a-2(^-i)y 



(87) 



(7^ 



> 2^ (88) 



>cia^^(-log<7)^-i (89) 



Case 2) '7 < « < > for ^ > 1: 
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In this case, we bound the output distortion by the average distortion caused by a large jump to 
another segment. Let zi be the additive noise in the first dimension and f{x) = s the modulated 
vector corresponding to the source sample x. 

For any point in the interval — | + (fc — l)a~^ < x < + ka"^ (for 1 < A; < a — 2'+^), 
when zi > 2'+^a^\ for any point x' < x + 2'a~^, the received point f{x) + z is closer to 
/ (x' + 2'a^^) than f{x). Therefore, the decoded signal is 5; > a; + 2'a^^ Thus, in this case, 
the squared error is at least (2^a~^)^. Therefore, the average distortion is lower bounded by 



D>Fr 



i<.<l-2-a-}. 



PT{z^>2'+'a~'}■{2'a-'Y 



(90) 




(91) 




(92) 




(93) 



By using 



e-^' < Q{x), 




(94) 




(95) 



By combining the bounds (for two cases), and noting that cr^ < 



2' 



D > min 



{c,a'{-\oga),c,a'''{-\ogaf-'} 



(96) 



£)>cV^(-l0g(7) 



N-1 



for N>2. 



(97) 
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Appendix C: Coding for unbounded sources 

Consider {^j}^^ as an arbitrary memoryless i.i.d source. We show that the results of Section 
V can be extended for non-uniform sources, to construct robust joint-source channel codes with 
a constraint on the average power. Without loss of generality, we can assume the variance of 
the source to be equal to 1 . For the source sample x, we can write it as a; = xi + X2 where xi 
is an integer, — | < X2 < |, and X2 + | = (O ■ 6162^3 •••) 2- Now, we construct the A^-dimensional 
transmission vector as s' = {s[, s'2, s'j^) = (xi + si — i, S2 — |, sn — where si, sn 
are constructed using (l36l) in section V. Let Di be the distortion conditioned on correct decoding 
of xi. Similar to the proof Theorem 6, we can show that the Di is upper bounded by 



Di < cicr2^2"2V-iog<x (9g) 

where ci and C2 depend only on A^. 

Now, we bound the distortion D2, for the case that xi is not decoded correctly. Since si is 



constructed by scheme II (in Section V), si is between and (0.10111 ■ ■ ■)2, hence < Si < |. 
To have an error of \xi — Xi\ = k, the amplitude of the noise on the first dimension should 
be greater than hence its probability is bounded by 2Q f j. When |xi — xi| = k, the 
overall squared error is lower bounded by 

Ix — x| < |xi — xi| + |x2 — 5;2| < k + 1. (99) 
Therefore, by using the union bound for all values of k, the distortion D2 is lower bounded by 

D2<Yl2Q\^-^j (k + l) (100) 



3\2 



00 

< ^e' ' 2 ' -{k + l) (101) 

k=l 

<C3e^. (102) 



Thus, D <Di + D2< C4a2^2"2V-ioga_ 



29 



To finish the proof, we only need to show that the average transmitted power is bounded. For 



S2, s^, the transmitted power is bounded as < j. For s[. 



J 1 2 



Xi + Si- 



< I \xi\ 

2 



Si 



< 1 1^1 + ^ + ^ ) = + 



Thus, using the Cauchy-Schwarz inequality, 

Ejls^l'} <E{(|x| + l)'} < (v^^+l)' 



< + 



4. 



(103) 
(104) 

(105) 
(106) 



Appendix D: Proof of Theorem 7 



We consider two cases for a, the scaling factor. 



Case 1) a < 2 



2(iV-M) 



log a) 



-(iV-Af) 

2JVI : 



Each subset of the modulated signal set is the scaled version of a segment of the source signal 
set by a factor of a, hence, we can lower bound the distortion by only considering the case that 
the subset is decoded correctly and there is no jump to adjacent subsets. 



> E ||x — x|^|no jump^ 



(107) 



(108) 



2JV N-M 

> CiG M (— log cr) M 



, , -1 , 2(N-M) 

Case 2) < 



(iV-M) -(JV-M) 
" H (- log 0-) 2H 



< for ; > 3: 



(109) 



In this case, we bound the output distortion by the average distortion caused by a jump to 
another subset. Without loss of generalitjzi we can consider a < (^), hence 2^'a > 8. First, we 

7 1 1 2iV . , N — M 

For 1 < a < -, the distortion D is larger than Di (the distortion for a = -), hence D > Di > Dig Jv/ (— logcr) , 

^ e ^ e e 

and Di depends only on A^. 
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show that there are two constants C5 and cq (independent of a and a) such that probability of 



an squared error of at least C5 (2 'a) ^ is lower bounded by 

Pr{jump) > CqQ logcr^ > cecr (110) 

By considering the power constraint, the maximum distance of each source sample to its quan- 
tization point is upper bounded by 

dmax < -■ (111) 

a 

We can partition the M-dimensional uniform source to n = ([|rj)^ > (2^)*^ cubes of 
size s = -r-i-r > — > 2'- dmax- Wc considcr Bi as the union of the quantization regions whose 
center is in the zth cube (1 < i < n). Because the decoding of digital and analog parts are 
done separately, the {N — M) -dimensional subspace (dedicated to send the quantization points) 
can be partitioned to n decoding subsets, corresponding to regions Bi, If we consider 

Ci, ...,Cn, the intersections of these decoding regions and the (AT — M) -dimensional cube of size 
4, centered at the origin, at least ^ of them have volume less than 2 — < — ncr < 

6 ' 2 \ n J - " 

2a^~^{— log a)^2—. This volume is less than the volume of an (iV — M) -dimensional sphere 
of radius a{— logcr)^. Thus, for any point inside Bi with this property, the probability of being 
decoded to a wrong subset Bj is at least equal to the probability that the amplitude of the noise 
is larger than the radius of that sphere (i.e. a{—loga)^). This probability is lower bounded 
by Pr l^;! > o-(— loga)^| = Q [^/— log a) > a. Now, for the cubes corresponding to these 
subsets, we consider points inside a smaller cube of size |, with the same center. 

For these points, at least with probability a, decoder finds a wrong quantization region where 
the distance of its center and the original point is at least = f ^ hence, the final 
squared error is at least - dmax^ > - > C5 (2"'a) ^ 

Because at least half of the n subsets have the mentioned property, the overall probability of 
having this kind of points as the source is at least |2~^, and in transmitting these points, with 
a probability which is lower bounded by a, the squared error is at least C5 (2~'a) ^. Therefore, 
the distortion is lower bounded by 



D > -a-c^ (2"'«)"' > C7(7 (2-'a)"' 
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2(N-M) 

> Cscr ■ a M 




2JV-M 



CgO" M 



(112) 



Finally, by considering the minimum of ( 11091 ) and (II 121) . we conclude 



(— logcr) *f 



N-M 



D > ca M 



(113) 
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